跳到主要内容

k8s 二进制 安装

https://blog.csdn.net/ljx1528/article/details/108465272

https://blog.csdn.net/kanganrui/article/details/105936821

https://www.cnblogs.com/zhaobin-diray/p/13724988.html

https://www.yuque.com/grep/kubernetes/cdiy10#XkUtC

https://blog.csdn.net/weixin_39773393/article/details/108330805

签发client证书（apiserver和etcd通信证书） k8s-api-client-csr.json

{
    "CN": "k8s-node",
    "hosts": [
    "192.1"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "od",
            "OU": "ops"
        }
    ]
}


/opt/cfssl/cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client k8s-api-client-csr.json |/opt/cfssl/cfssl-json -bare k8s-api-client

签发server证书（apiserver和其它k8s组件通信使用）

k8s-api-server-csr.json

{
    "CN": "k8s-apiserver",
    "hosts": [
        "127.0.0.1",
        "192.168.0.1",
        "kubernetes.default",
        "kubernetes.default.svc",
        "kubernetes.default.svc.cluster",
        "kubernetes.default.svc.cluster.local",
        "10.4.7.10",
        "10.4.7.21",
        "10.4.7.22",
        "10.4.7.23"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "od",
            "OU": "ops"
        }
    ]
}

/opt/cfssl/cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client k8s-api-server-csr.json |/opt/cfssl/cfssl-json -bare k8s-api-server

![](69747A12BE915FCCB85A68EEE6E60F4F.jpg)

##Kubernetes API Server

vim /usr/lib/systemd/system/kube-apiserver.service


[Unit]

Description=Kubernetes API Server

Documentation=https://github.com/kubernetes/kubernetes

After=etcd.service

Wants=etcd.service

[Service]

EnvironmentFile=/opt/k8s/kubu-api-start.sh

ExecStart=/bin/bash /opt/k8s/kubu-api-start.sh

Restart=on-failure

Type=notify

[Install]

WantedBy=multi-user.target

vim /opt/k8s/kube-apiserver-start.sh 启动命令 配置

#!/bin/bash

/opt/k8s/kube-apiserver  --logtostderr=true \
--v=2 \
--log-dir=/opt/k8s/logs \
--etcd-servers=https://192.168.137.132:2379,https://192.168.137.133:2379,https://192.168.137.134:2379 \
--bind-address=192.168.137.132 \
--secure-port=6443 \
--advertise-address=192.168.137.132 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--enable-bootstrap-token-auth=true \
--token-auth-file=/opt/k8s/conf/token.csv \
--service-node-port-range=30000-32767 \
--kubelet-client-certificate=/opt/ssl/k8s-api-client.pem \
--kubelet-client-key=/opt/ssl/k8s-api-client-key.pem \
--tls-cert-file=/opt/ssl/k8s-api-server.pem  \
--tls-private-key-file=/opt/ssl/k8s-api-server-key.pem \
--client-ca-file=/opt/ssl/ca.pem \
--service-account-key-file=/opt/ssl/ca-key.pem \
--etcd-cafile=/opt/ssl/ca.pem \
--etcd-certfile=/opt/ssl/etcd.pem \
--etcd-keyfile=/opt/ssl/etcd-key.pem \
--audit-log-maxage=30 \
--audit-log-maxbackup=3 \
--audit-log-maxsize=100 \
--audit-log-path=/opt/k8s/logs/k8s-audit.audit



参数解析：

--etcd-servers=http://127.0.0.1:2379 

etcd的地址，若etcd是集群，则配置集群所有地址，用逗号隔开

--insecure-bind-address=0.0.0.0

aipServer的监听地址，默认为127.0.0.1，若要配置集群，则要设置为0.0.0.0才能被其他主机找到

--insecure-port=8080 

apiserver的监听端口

--service-cluster-ip-range=169.169.0.0/16 

service的地址范围，用于创建service的时候自动生成或指定serviceIP使用

--service-node-port-range=1-65535 

如果定义service时（后面的nginx-svc.yaml）将type字段设置为NodePort，kubernetesmaster将会为service的每个对外映射的port分配一个”本地port“，这个本地port作用在每个node上，且必须符合定义在配置文件中的port范围（为–service-node-port-range）。

--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,DefaultStorageClass,ResourceQuota 

使用的系统组件，具体组件的作用参考官网，原来还有一个serviceAccount，因为我们没有配置serviceAccount的参数，所以这里去掉了，采用非安全连接的方式。

--logtostderr=true 
日志默认存储方式，默认存储在系统的journal服务中

--v=2

日志等级，0是debug，2是error


启用 TLS Bootstrapping 机制
TLS Bootstraping：Master apiserver启用TLS认证后，Node节点kubelet和kube-proxy要与kube-apiserver进行通信，必须使用CA签发的有效证书才可以，当Node节点很多时，这种客户端证书颁发需要大量工作，同样也会增加集群扩展复杂度。为了简化流程，Kubernetes引入了TLS bootstraping机制来自动颁发客户端证书，kubelet会以一个低权限用户自动向apiserver申请证书，kubelet的证书由apiserver动态签署。所以强烈建议在Node上使用这种方式，目前主要用于kubelet，kube-proxy还是由我们统一颁发一个证书。
TLS bootstraping 工作流程：

![IMAGE](resources/C867F549D2EA6F62DED55DDFD829673A.jpg =666x578)

创建上述配置文件中token文件：

cat   /opt/kubernetes/cfg/token.csv << EOF
c47ffb939f5ca36231d9e3121a252940,kubelet-bootstrap,10001,"system:node-bootstrapper"
EOF

格式：token，用户名，UID，用户组
token也可自行生成替换：
head -c 16 /dev/urandom | od -An -t x | tr -d ' '

授权kubelet-bootstrap用户允许请求证书

kubectl create clusterrolebinding kubelet-bootstrap \
--clusterrole=system:node-bootstrapper \
--user=kubelet-bootstrap

##kube-controller-manager服务 1.2 配置systemd服务文件 vim /usr/lib/systemd/system/kube-controller-manager.service

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvironmentFile=/opt/k8s/conf/kube-controller-manager.conf
ExecStart=/opt/k8s/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

1.2.2 配置controller-manager vim /opt/k8s/conf/kube-controller-manager.conf

cat > /opt/k8s/conf/kube-controller-manager.conf << EOF
KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=true \\
--v=2 \\
--log-dir=/opt/k8s/logs \\
--leader-elect=true \\
--master=127.0.0.1:8080 \\
--bind-address=127.0.0.1 \\
--allocate-node-cidrs=true \\
--cluster-cidr=10.244.0.0/16 \\
--service-cluster-ip-range=10.0.0.0/24 \\
--cluster-signing-cert-file=/opt/ssl/ca.pem \\
--cluster-signing-key-file=/opt/ssl/ca-key.pem  \\
--root-ca-file=/opt/ssl/ca.pem \\
--service-account-private-key-file=/opt/ssl/ca-key.pem \\
--experimental-cluster-signing-duration=87600h0m0s"
EOF

参数解析：

--master

监听主机IP地址，0.0.0.0监听主机所有主机接口

##kube-scheduler服务 1.3.1 配置systemcd服务文件

vim /usr/lib/systemd/system/kube-scheduler.service

[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvironmentFile=/opt/k8s/conf/kube-scheduler.conf
ExecStart=/opt/k8s/kube-scheduler \$KUBE_SCHEDULER_OPTS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

1.3.2 配置kube-scheduler vi /opt/k8s/conf/kube-scheduler.conf

cat > /opt/k8s/conf/kube-scheduler.conf << EOF
KUBE_SCHEDULER_OPTS="--logtostderr=true \
--v=2 \
--log-dir=/opt/k8s/logs \
--leader-elect \
--master=127.0.0.1:8080 \
--bind-address=127.0.0.1"
EOF

启动​

完成以上配置后，按顺序启动服务

systemctl daemon-reload systemctl enable kube-apiserver.service systemctl start kube-apiserver.service systemctl enable kube-controller-manager.service systemctl start kube-controller-manager.service systemctl enable kube-scheduler.service systemctl start kube-scheduler.service

检查每个服务的健康状态：

systemctl status kube-apiserver.service systemctl status kube-controller-manager.service systemctl status kube-scheduler.service

##查看集群状态

###生成kubectl连接集群的证书 kubelet-admin-csr.json

{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}

/opt/cfssl/cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer kubelet-admin-csr.json | /opt/cfssl/cfssl-json -bare kubelet-admin

生成kubeconfig文件：

KUBE_CONFIG="/opt/k8s/conf/kube.yml"
KUBE_APISERVER="https://192.168.137.132:6443"

/opt/k8s/kubectl config set-cluster kubernetes \
  --certificate-authority=/opt/ssl/ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=${KUBE_CONFIG}
/opt/k8s/kubectl config set-credentials cluster-admin \
  --client-certificate=/opt/ssl/kubelet-admin.pem \
  --client-key=/opt/ssl/kubelet-admin-key.pem \
  --embed-certs=true \
  --kubeconfig=${KUBE_CONFIG}
  
/opt/k8s/kubectl config set-context default \
  --cluster=kubernetes \
  --user=cluster-admin \
  --kubeconfig=${KUBE_CONFIG}
/opt/k8s/kubectl config use-context default --kubeconfig=${KUBE_CONFIG}
所有组件都已经启动成功，通过kubectl工具查看当前集群组件状态：

kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok                  
controller-manager   Healthy   ok                  
etcd-2               Healthy   {"health":"true"}   
etcd-1               Healthy   {"health":"true"}   
etcd-0               Healthy   {"health":"true"}